Fusion of magnetic resonance and ultrasound images for endometriosis detection

Endometriosis is a gynecologic disorder that typically affects women in their reproductive age and is associated with chronic pelvic pain and infertility. In the context of pre-operative diagnosis and guided surgery, endometriosis is a typical example of pathology that requires the use of both magnetic resonance (MR) and ultrasound (US) modalities. These modalities are used side by sidebecause they contain complementary information. However, MRI and US images have different spatial resolutions, fields of view and contrasts and are corrupted by different kinds of noise, which results in important challenges related to their analysis by radiologists. The fusion of MR and US images is a way of facilitating the task of medical experts and improve the pre-operative diagnosis and the surgery mapping. The object of this PhD thesis is to propose a new automatic fusion method for MRI and US images. First, we assume that the MR and US images to be fused are aligned, i.e., there is no geometric distortion between these images. We propose a fusion method for MR and US images, which aims at combining the advantages of each modality, i.e., good contrast and signal to noise ratio for the MR image and good spatial resolution for the US image. The proposed algorithm is based on an inverse problem, performing a super-resolution of the MR image and a denoising of the US image. A polynomial function is introduced to modelthe relationships between the gray levels of the MR and US images. However, the proposed fusion method is very sensitive to registration errors. Thus, in a second step, we introduce a joint fusion and registration method for MR and US images. Registration is a complicated task in practical applications. The proposed MR/US image fusion performs jointly super-resolution of the MR image and despeckling of the US image, and is able to automatically account for registration errors. A polynomial function is used to link ultrasound and MR images in the fusion process while an appropriate similarity measure is introduced to handle the registration problem. The proposed registration is based on a non-rigid transformation containing a local elastic B-spline model and a global affine transformation. The fusion and registration operations are performed alternatively simplifying the underlying optimization problem. The interest of the joint fusion and registration is analyzed using synthetic and experimental phantom images

La fusion de l'IRM et de l'échographie

L'endométriose est un trouble gynécologique qui touche généralement les femmes en âge de procréer et qui est associé à des douleurs pelviennes chroniques et à l'infertilité. L'endométriose est un exemple typique de pathologie qui nécessite l'utilisation de l'imagerie à résonance magnétique (IRM) et 'imagerie ultrasonore (US) (appelée aussi échographie) pour le diagnostic préopératoire et la chirurgie guidée. Ces modalités sont utilisées conjointement car elles contiennent des informations complémentaires. Cependant, le fait qu'elles aient des résolutions, des champs de vue et des contrastes différents et qu'elles soient corrompues par des bruits de differentes natures rend la collecte d'informations à partir de ces modalités difficile pour les radiologues. Ainsi, la fusion des images IRM et l'échographie peut faciliter la tâche des experts médicaux et améliorer le diagnostic préopératoire et le plan de l'intervention chirurgicale. L'objet de cette thèse de doctorat est de proposer une nouvelle méthode de fusion automatique des images IRM et US. Tout d'abord, nous supposons que les images IRM et US à fusionner sont alignées, c'est-à-dire qu'il n'y a pas de déformation géométrique entre elles. Nous proposons alors dans ce contexte idéal des méthodes de fusion pour ces deux images, qui visent à combiner les avantages de chaque modalité, c'est-à-dire un bon contraste et un bon rapport signal/bruit pour l'IRM et une bonne résolution spatiale pour l'échographie. L'algorithme proposé est basé sur un problème inverse, réalisant une super-résolution de l'image IRM et un débruitage de l'image US. Des fonctions polynomiales sont introduites pour modéliser les relations entre les niveaux de gris des images IRM et US. Cependant, la méthode de fusion proposée est très sensible aux erreurs de recalage. C'est pourquoi, dans un deuxième temps, nous proposons une méthode conjointe de fusion et de recalage pour ces deux modalités. La fusion d'images IRM/US proposée permet d'obtenir conjointement une super-résolution de l'image IRM et un débruitage de l'image US, et peut automatiquement prendre en compte les erreurs de recalage. Une fonction polynomiale est utilisée pour relier les images ultrasonores et IRM dans le processus de fusion, tandis qu'une mesure de similarité appropriée est introduite pour traiter le problème de recalage. Le recalage proposé est basé sur une transformation non rigide contenant un modèle élastique local de Bspline et une transformation affine globale. Les opérations de fusion et de recalage sont effectuées alternativement, ce qui simplifie le problème d'optimisation sous-jacent. L'intérêt de la fusion et du recalage conjoints est analysé à l'aide des images synthétiques et expérimentales.Endometriosis is a gynecologic disorder that typically affects women in their reproductive age and is associated with chronic pelvic pain and infertility. In the context of pre-operative diagnosis and guided surgery, endometriosis is a typical example of pathology that requires the use of both magnetic resonance (MR) and ultrasound (US) modalities. These modalities are used side by sidebecause they contain complementary information. However, MRI and US images have different spatial resolutions, fields of view and contrasts and are corrupted by different kinds of noise, which results in important challenges related to their analysis by radiologists. The fusion of MR and US images is a way of facilitating the task of medical experts and improve the pre-operative diagnosis and the surgery mapping. The object of this PhD thesis is to propose a new automatic fusion method for MRI and US images. First, we assume that the MR and US images to be fused are aligned, i.e., there is no geometric distortion between these images. We propose a fusion method for MR and US images, which aims at combining the advantages of each modality, i.e., good contrast and signal to noise ratio for the MR image and good spatial resolution for the US image. The proposed algorithm is based on an inverse problem, performing a super-resolution of the MR image and a denoising of the US image. A polynomial function is introduced to modelthe relationships between the gray levels of the MR and US images. However, the proposed fusion method is very sensitive to registration errors. Thus, in a second step, we introduce a joint fusion and registration method for MR and US images. Registration is a complicated task in practical applications. The proposed MR/US image fusion performs jointly super-resolution of the MR image and despeckling of the US image, and is able to automatically account for registration errors. A polynomial function is used to link ultrasound and MR images in the fusion process while an appropriate similarity measure is introduced to handle the registration problem. The proposed registration is based on a non-rigid transformation containing a local elastic B-spline model and a global affine transformation. The fusion and registration operations are performed alternatively simplifying the underlying optimization problem. The interest of the joint fusion and registration is analyzed using synthetic and experimental phantom images

On the design of a pelvic phantom for magnetic resonance and ultrasound image fusion

International audienceThe purpose of this paper is to introduce a customized multi- modality phantom designed to facilitate the proof-of-concept of MRI/ultrasound fusion approaches. Phantom experiments are often required before in vivo validation, giving access to more challenging data than numerical simulations. Nevertheless, manufactured phantoms are expensive and usually lack of flexibility. In contrast, the proposed model was inexpensive and accurately designed to overcome multimodal registration issues

Magnetic Resonance and Ultrasound Image Fusion Using a PALM Algorithm

International audienceThis paper studies a new fusion algorithm for magnetic resonance (MR) and ultrasound (US) images combining two inverse problems for MR image super-resolution and US image despeckling. A polynomial function is used to link the gray levels of the two imaging modalities. Qualitative and quantitative evaluations on experimental phantom data show the interest of the proposed algorithm. The fused image is shown to take advantage of both the good contrast and high signal to noise ratio of the MR image and the good spatial resolution ofthe US image

Fusion Of Magnetic Resonance And Ultrasound Images: A Preliminary Study On Simulated Data

International audienceWe propose a new fusion method for magnetic resonance imaging (MRI) and ultrasound (US) data combining two inverse problems: MRI reconstruction using super-resolution and US image despeckling, using a model relating the two modalities through an unknown polynomial function. We demonstrate the accuracy of the proposed fusion algorithm by quantitative and qualitative evaluation using synthetic data. The resulting fused image is shown to have an improved signal to noise ratio and spatial resolution compared to the native MRI and US images

Ultrasound and magnetic resonance image fusion using a patch-wise polynomial model

International audienceThis paper introduces a novel algorithm for the fusion of magnetic resonance and ultrasound images, based on a patch-wise polynomial model relating the gray levels of the two imaging systems (called modalities). Starting from observation models adapted to each modality and exploiting a patch-wise polynomial model, the fusion problem is expressed as the minimization of a cost function including two data fidelity terms and two regularizations. This minimization is performed using a PALM-based algorithm, given its ability to handle nonlinear and possibly non-convex functions. The efficiency of the proposed method is evaluated on phantom data. The resulting fused image is shown to contain complementary information from both magnetic resonance (MR) and ultrasound (US) images, i.e., with a good contrast (as for the MR image) and a good spatial resolution (as for the US image)

Fusion of Magnetic Resonance and Ultrasound Images for Endometriosis Detection

International audienceThis paper introduces a new fusion method for magnetic resonance (MR) and ultrasound (US) images, which aims at combining the advantages of each modality, i.e., good contrast and signal to noise ratio for the MR image and good spatial resolution for the US image. The proposed algorithm is on an inverse problem, performing a super-resolution of the MR image and a denoising of the US image. A polynomial function is introduced to model the relationships between the gray levels of the MR and US images. The resulting inverse problem is solved using a proximal alternating linearized minimization algorithm. The accuracy and the interest of the fusion algorithm are shown quantitatively and qualitatively via evaluations on synthetic and experimental phantom data